This invention relates generally to the field of video display systems which present a changing visual image to the user, and in particular to such systems where aspects of the video display are controlled by the user. Even more particularly, the invention relates to such systems where the video image is controllable relative to the perceived rate of motion and direction of travel within the virtual display.
Interactive video games which are controlled by players observing a video monitor and reacting to video images displayed on the monitor, or games where future video images are controlled by commands made by the player, are well known. Such games are presented on self-contained play units which operate with game program cartridges, such as the popular units sold under the brand names NINTENDO or PLAYSTATION, or on personal computers where software is provided on CD-ROMs or diskettes, or downloaded from the Internet, or where play is accomplished through remote servers on the Internet. Control or movement of the video image is accomplished by movement or actuation of mechanical controls, such as buttons on a hand-held device, a joystick, a mouse or keys on the keyboard of a computer. These interactive video games, while providing visual and mental stimulation, require that the players remain essentially stationary or sedentary. A latter generation of video games, often found in arcades, use control mechanisms which simulate actual real world control mechanisms for various scenarios. For example, a steering wheel will be used to control the perceived direction of travel of the video image in a race car video game, while a simulated gas pedal will control the perceived speed of travel. In a motorcycle race video game, the user mounts a simulated motorcycle and controls direction by leaning the motorcycle left or right. Similar systems for simulated skiing, flying, etc. provide control of the video image by body movement.
The beneficial effects of exercise are well documented. Physical exertion which includes spatial movement in outdoor settings, such as jogging, bike riding, rowing, and the like, are very popular, but many if not most participants will not undertake the activity unless there is good weather, a safe environment, daylight, etc. This issue has been addressed in certain large gyms by providing an indoor track, which allows actual movement by the participants during the activity. Where there is limited space, such as in a home, stationary exercise equipment has been developed which allows the user to perform the same motions as the outdoor activity but without any actual change of location. To replace running, biking and rowing, for example, well known equipment such as treadmills, stationary bikes and rowing machines are used. A downside to the use of such stationary equipment is that the scenery remains static, and thus long term exercise sessions can be dull and monotonous. Many users will read while exercising on stationary equipment, or televisions are set up to provide a mental distraction.
It has been found desirable to combine the positive aspects of video games with the positive aspects of physical movement or exercise in a stationary location, such that a person performing exercises is presented with a stimulated video image to reduce boredom, or such that for a person playing a video game the perceived speed of travel is controlled by sensing body movement. In addition, the combination of body movement, visual stimulation and mental stimulation can be used in educational or training games to combat boredom and short attention spans.
One approach to accomplish these goals is to provide a changing video display which is connected to and responsive to motion of a particular piece of exercise equipment. For example, U.S. Pat. No. 5,591,104 to Andrus et al., U.S. Pat. No. 6,004,243 to Ewert, and U.S. Pat. No. 6,024,675 to Kashiwaguchi all show stationary bicycles which are electronically connected to a computer and video display monitor, where the rate of perceived motion of the video image is controlled by the pedaling rate. In U.S. Pat. No. 5,240,417 to Smithson et al. and U.S. Pat. No. 5,462,503 to Benjamin et al., a stationary bike where the perceived direction of travel is controlled by leaning is shown. Similar systems can be used with rowing machines, flying machines and treadmills, such as shown in U.S. Pat. No. 5,385,519 to Hsu et al., U.S. Pat. No. 5,489,249 to Brewer et al., U.S. Pat. No. 5,562,572 to Carmein, and U.S. Pat. No. 5,584,700 to Feldman et al. More complicated virtual reality systems are also known, such as shown in U.S. Pat. No. 5,577,981 to Jarvik. While these devices are each an improvement over their respective non-video equivalents, the systems are specific to a particular piece of equipment and not interchangeable, such that the video display system for a stationary bike cannot be transferred to a treadmill or a rowing machine by the user.
Especially for home-use exercise situations, it is most desirable that the interactive changeable video image display system be responsive to body movement through sensing devices attached directly to the user rather than to the exercise equipment itself. This allows the system to be utilized with different types of equipment, or with no equipment at all, such as where the user simply runs in place. Likewise, where the interactive changeable video image display system responsive to movement is used with entertainment or educational games, it is most desirable that the system not have required hardware specific only to a single game.
An example of a system where the sensing devices are positioned directly onto the user""s body is shown in U.S. Pat. No. 5,524,637 to Erickson. Erickson discloses an accelerometer connected to the user""s ankle, means for wireless transmission of information from the sensor to a computer, software to interpret the information and control a video display image, and a monitor to present the image to the user. The perceived rate and direction of travel are responsive to the rate and direction of movement of the user. Erickson""s invention is a device for measuring exercisexe2x80x94it calculates and records very specific aspects of exercise via its accelerometer. It is not a device for general, casual interaction with the computer for game play, nor does it substitute for a keyboard, mouse, joystick, steering wheel, etc., but rather is a device to perform functions and measurements that the keyboard, mouse, etc., cannot do. In contrast to Erickson""s system, this invention is not a device for measuring exercise and it is in fact a device for general, casual interaction with the computer for game play and other all-purpose use, including educational use for children who are blind or deaf. The Erickson system is limited to its applicability to various exercises and particularly is limited to its applicability to various games, since the perceived direction control is responsive only to physical directional changes.
It is an object of this invention to provide an interactive video image display system which is responsive to both body motion information and verbal information, where the body motion information is used to control the perceived rate of travel through the virtual world of the video image, and where the verbal information is used to control the perceived rate of direction as well as to control an unlimited number of virtual actions within the video image which have been programmed into the operating software. Voice recognition software and the requisite equipment to accomplish this are well known in the art.
Unlike Erickson""s system, the invention is not designed to monitor or measure exercise, but is a device to enable interaction with the computer via the combination of physical exertion and voice commands. The invention has uses far beyond that of the known exercise-type devices. For example, a child""s software game that takes place in a virtual maze can be played with the child walking in place and telling the computer which way to turn in the maze by verbal commands. Actions such as jump, hyperspace, duck, blast off, freeze, shoot, swim, stop, go, pause, play, back, next, help, save, shift, land, walk, run, brake, fast, slow, hide, seek, etc. can be programmed to be responsive to voice commands, as well as to control manipulation of virtual options, such as go to X, pick up X, etc.
When used to enhance the home exercise experience, the system converts any simple stationary exercise equipment into a virtual reality machine, the user now being able to travel through a virtual reality world displayed on the monitor of any multimedia computer. The virtual worlds or games exist on CD-ROM or any other software storage device, or at a website accessible over the Internet, and can range from futuristic racetracks to farmlands and woods and neighborhoods to underwater worlds to mountains to intergalactic adventures to any imaginable world or landscape or setting. In each virtual world, a person can move about by exercising or body movement and voice command. The voice commands will vary with each application, with a simple system utilizing the commands xe2x80x9cleftxe2x80x9d, xe2x80x9crightxe2x80x9d and xe2x80x9cbackxe2x80x9d, where the first two cause perceived movement of 45 degrees to either side and the last caused perceived movement of 180 degrees or a reversal of view. Object commands such as xe2x80x9cboatxe2x80x9d, xe2x80x9ccarxe2x80x9d, xe2x80x9cplanexe2x80x9d, xe2x80x9crocketxe2x80x9d, xe2x80x9cladderxe2x80x9d, xe2x80x9cbridgexe2x80x9d, xe2x80x9ctunnelxe2x80x9d, xe2x80x9cshovelxe2x80x9d, xe2x80x9cdrillxe2x80x9d, xe2x80x9cswordxe2x80x9d, xe2x80x9criflexe2x80x9d, etc. could provide access to vehicles and tools necessary to access certain parts of the virtual worlds in a manner well known to game players.
When used in conjunction with a child""s game, the invention enables interaction with the game environment. This means that for an educational software game, the game actually combines physical activity and learning. For example, the game MazeQuest: Tales of the Wandering Grammarian, by Ohio Distinctive Software, properly adapted for use as part of this invention, would involve the child giving voice commands and walking in place to travel through the virtual maze, as well as the child practicing and learning grammar by providing voice answers to grammar questions, which could be presented in a multiple choice format. As another example, in a child""s painting software program the child""s movements combined with voice commands (e.g., to select colors and art tools) could enable an entirely new kind of artistic expression. Because many children have difficulty sitting still to learn, and virtually all children have attention spans which are relatively limited, an educational game that utilizes the capabilities of this invention can be an extraordinary learning device.
The invention also opens up the world of educational games to children who are blind. The invention enables interaction with the computer via a combination of body movement and voice commands such that there is no need for movement of a mouse or depression of keys responsive to visually displayed information. An educational game for blind children might be based on a virtual journey through unlit caverns, a night expedition or an Antarctic whiteout. The child would navigate based on audio clues from the game, such as the sound of footsteps on a correct stone path versus footsteps in water, mud, dense undergrowth, etc. Audio clues could be used to indicate dead ends in maze situations. Still another example could be to follow a winding stream based on the sound of the water, with splashes indicating a misstep, or a hide-and-seek game where the object is to move away from the sound of pursuers, or a nighttime sea rescue based on directional clues combined with positive and negative audio sounds (e.g., warning bells, sonar depth warnings, radio dispatches, shouts, etc.), or a nighttime tracking mission of an animal with a locator transmitting beacon. The educational component for blind children would be the same as for sighted children, that upon achieving certain goals or upon encountering certain objects or creatures in the game, the computer would ask multiple choice questions relating to math, spelling, geography, language, science, history, etc., with the child verbally responding.
For children with speech problems or other difficulties in speaking, such as with deaf children, the invention can learn to recognize the child""s speech in the known manner of standard voice recognition programs. Thus the invention can be used as a speech practicing tool while the child is simultaneously having fun and learning. The instant positive reinforcement that the child would experience by the game responding to his or her voice commands would encourage the child to practice speaking.
These and other objects of the invention not particularly set forth above will be more readily apparent from the description and disclosure to follow.
The invention is an interactive video display and computer system which provides changing video images in response to a combination of signals received from repetitive body movements and voice commands. The system comprises a pace sensing means which is worn on the user""s body which senses the repetitive body motion, translates that motion into a signal and transmits the pace signal to a signal receiving means, which translates the signal into a signal readily recognized by a computer and then delivers the signal to the computer system. The system further comprises a voice receiving means, which can operate in either of two ways. A first scenario is that the voice receiving means consists of both a microphone and the necessary hardware and software to translate the voice input into signals readily understood by the computer (e.g., signals identical to keyboard inputs, mouse inputs or any other generally recognized computer inputs). The voice receiving means receives the voice command, translates it into a generally recognized computer input, and then sends that input to the computer. In a second scenario, the voice receiving means contains a microphone, but the hardware and software to translate the voice into input that can be readily recognized by the computer would reside in the computer system itself. In this second scenario, the voice receiving means receives the voice command and then sends the untranslated input to the computer for translation. The advantage of the first scenario is that it is more universally accessible because it does not require that the user""s computer have any particular microphone translation capabilities. The disadvantage is that the cost of the device under the first scenario is greater than the cost under the second scenario because the first scenario contains more hardware and software. The computer system receives and analyzes the pace signals and the voice signals, and in turn controls a streaming video display image presented on a monitor, such that the perceived rate of travel within the virtual world presented by the video display system is directly responsive to the frequency of the pace signal, and where the perceived direction of travel within the virtual world presented by the video display system is directly responsive to the voice signals. In other embodiments, the voice signals can be used by the software to manipulate the video display in other ways concerning the particular images presented to the user. In a basic embodiment, the pace sensing means may comprise a pendulum switch whereby a single pulse is produced for each repetitive motion, such as back-and-forth movement of an arm or leg which would result from running in place or operation of an exercise device.